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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re International Application of 

International Serial No. PCT/ JPOO/00872 

International filing date: February 16, 2000 

For: Speech Transformation Method and Apparatus 

VERIFICATION OF TRANSLATION 

Honorable Commissioner of Patents and Trademarks 

Washington, D.C. 20231 

Sir: 

Eiichi Tamura, a member of A.KOIKE & CO., of ll~Mori 
Bldg., 6-4, Toranomon 2-chome, Minato-ku, Tokyo 105-0001 Japan, 
declares : 

(1) that he knows well both the Japanese and English 
languages; 

(2) that he translated the above-identified International 
Application from Japanese to English; 

(3) that the attached English translation is a true and 
correct translation of the above- identi f led International 
application to the best of his knowledge and belief; and 

(4) that all statements made of his own knowledge are 
true and that all statements made on information and belief are 
believed to be true, and further that these statements are made 
with the knowledge that willful false statements and the like are 
punishable by fine or imprisonment, or both, under 18USC 1001, 
and that such false statements may jeopardize the validity of the 
application or any patent issuing thereon. 




Date 



Eiichi Tamura 
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Claims 




1. A speech tran^ormation apparatus comprising: 

acousto-electric transducing means for detecting the speech to generate speech 
signals; 



X 



recognition rneans for perforaiing speech recognition processing using the 



speech signals from said acousto-electric transducing means; 

transforming means for working on and transforming the result of recognition 
from said recognition n^eans depending on the bodily state, the using state or the using 
objectives of a user; 

output control meWs for generating a control signal for outputting the result 
recognized by said recognition means and/or the result of recognition obtained on 
working and transfonnatiqn operations by said transfonning means; and 

output means for c^utputting the result of recognition recognized by said 



recognition means and wor 
on said control signal ge 



ied on and transformed by said transfonning means based 
erated by said control means to present the result of 



recognition to the user. 



1 



2. The speech transfonnatibnapparatus according to claim 1 wherein said 
acousto-electric transducing means^gi^erat-e^ uttered with voice and speech 



disorder to generate speech signalsVapd wherei 

said transforming means inclu^s\pee(^ recognition means for perfonning 
processing for speech recognition based on sjaeech signals from said acousto-electric 
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transducing means, storage means for memorizing speech data generated on previously 
sampling the speech uttered without voice-speech disorders and speech information 
generating means for generafang the speech information indicating the output speech, 
using the speech data memorized in said storage means, based on the result of 
recognition by said speech recognition means. 

3. The speech transformation apparatus according to claim 1 wherein said output 
means is display means for demonstrating an image; and wherein 

said output control means generates a control signal so that the result of 
recognition worked on and transflormed and/or the result of recognition not worked on 
or transformed is demonstrated image on display means of said output means. 

4. The speech transfonna\((^ apbaj-atus according to claim 3 wherein said output 
control means includes data s^bragp means having stored therein data indicating a 
pattern for demonstration on saidyflisplay means, and manages control, based on the 
result recognized by said recognitiommeans and/or the result of recognition obtained 
on working on or transfonning said recognized result by said transfonning means, for 
reading out data stored in said data storage means to demonstrate the pattern indicated 
by said data on said display means. 

5. The speech transformation apparatus according to claim 3 wherein said output 
means further includes electro-acoustic transducing means for outputting speech; and 
wherein 



said output control means generates a control signal for outputting from said 
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electro-acoustic transducing means the result of recognition and/or the result of 
recognition worked on and transformed as speech. 

6. The speech transformation apparatus according to claim 5 wherein said output 
control means generates a control signal so that an image will be demonstrated on said 
display means for the speech uttered by a user and/or a person other than the user, said 
output control means amphfying theteound pressure level of the speech uttered by the 
user and/or a person other than the u^er for outputting the sound as speech from said 
electro-acoustic transducing means. 

7. The speech transformation apparaks according to claim 3 wherein said output 
control means generates a control silgnal, re^^P^ive to the result of recognition, for 
demonstrating the meaning and tlf^/^ontents Jof the speech detected by said 
acousto-electric transducing means. 

8. The speech transformation apparatus kco6rding to claim 1 wherein said output 
means is a cochlear implant mechanism; ana wherein 

\ 

said output control means generates a control signal for outputting the result of 
recognition and/or the result of recognition worked on or transformed as electric 
signal. 

9. The speech transformation apparatus accorciing to claim 1 wherein said output 
control means is a diaphragm for directly imparting vibrations to said diaphragm; and 
wherein 

said output control means generates a control signal so that the result of 
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recognition and/or the result of recognition worked on and/or transformed will be 
outputted as electrical signal 

10\ The speech transformation apparatus according to claim 1 wherein said output 
mea^s is a cochlear implant mechanism and wherein 

isaid output control means generates a control signal for outputting the result of 
recogm^ion and/or the result of recognition worked on or transformed as electric 
signal. 

ll/^^I^e speech transformation apparatus according to claim 1 wherein said output 
means is an bone conduction ultrasonic mechanism. 

12. The speech transformation apparatus according to claim 1 wherein said output 
means is a vibratormay for a tactile aid and wherein 

said output contKol means generates a control signal so that the result of 
recognition and/or the resullMrecp^ition worked on or transfonned will be outputted 
to said vibrator array as electMca^ 

13. The speech transfonnation app^tus according to claim 1 wherein said output 
means is an auditory brainstem impl^t supplying electrical signals through an 
electrode to the cochlear nuclei and wherei 

said output control means generates a^control signal so that the result of 
recognition and/or the result of recognition worked os^ or transfonned will be outputted 
to said cochlear nuclei as electrical signal. 

14. The speech transformation apparatus according to c'kim 1 wherein said output 
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means is detachable with respk^t to the user. 

15. The speech transformation apparatus according to claim 1 wherein the speech is 
fed through a communication network to said acousto-electric transducing means and 
wherein said communication means outputs the result of recognition from said output 
means to said communication network 

16. The speech transformation apparatus according to claim 1 wherein said 
recognition means performs speaker recognition processing on the input speech to 
generate the result of recognition associated with each speaker; and wherein 

said output means presents the infonnS^tion pertinent to each speaker to the user. 

17. The speech transformation apparaO ac)&ei;ding to claim 2 further comprising: 

imaging means for photographilg^n^ 

said imaging means outputting the phopgraphed image at least to display 

means. 

1 8. The speech transformation apparatus according to claim 17 wherein said imaging 
means performs image transfonning processing onvthe photographed image based on 
the visual acuity of the user to output the transfornied image to said display means. 

19. The speech transformation apparatus according t\ claim 17 wherein said imaging 
means is detachable with respect to the user. 

20. The speech transformation apparatus accordiAg to claim 1 wherein said 
communication means is connected to an extemal equipi\ient contained in an extemal 
network; and wherein 
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data from an external equipment is outputted as result of recognition worked on 
or transformed depending oi^ the bodily state, the using state or the using objectives 
of the user. 

21. The speech transformation apparatus according to claim 20 wherein said 
communication means outputs speech signals, generated by said acousto-electric 
transducing means, the result om-ecognition obtained by said recognition means and/or 
the result of recognition as worked on and transformed by transforming means to an 
external equipment to obtain theVesult of recognition and/or the result of recognition 
as worked and transformed from said external equipment. 

22. The speech transformation Apparatus according to claim 20 wherein said 
communication means recei\^sYJpogram adapted for changing the processing 
contents of said recognition niea^s landA)r said working and transfonnatioii means 
from an external equipment; an<^herein 

said recognition means and/ or saia working and transformation means generates 
the result of recognition and/or the results of working and transformation operations 
based on said program received by said communication means. 

23. The speech transformation apparatus according to claim 1 wherein said output 
control means manages control to outpul the result of recognition and/or the 
transformed result of recognition simultaneously or with a time difference. 

24. The speech transfonnation apparatus according to claim 1 wherein said 
acousto-electric transducing means, recognitiommeans, transforming means, output 
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control means and the outjiut means are designed as respective plural devices, 
depending on the bodily state,\the using state or the using objectives of the user, and 
wherein 

the respective devices are^interconnected over a radio route to transmit/receive 
at least the result of recognition^d/or the transformed result of recognition. 

25. The speech transformatiom apparatus according to claim 2 wherein said 
acousto-electric transducing meansmetects the speech uttered using auxihary means 
or devices used for correcting voice-speech disorders, as the speech uttered by a 
person with^ice-speech disorders, to generate speech signals. 

26. The speech transformation apparatus according to claim 25 wherein said 

. \ . 

acousto-electric trans4ucing means detects the speech uttered using speech production 
substitutes, as the speecKWered by a persbn with voice-speech disorders, to generate 
speech signals. 

27. ThA speech transfomiation apparatus according to claim 25 wherein said 

acousto-eloctric transducing means detects the speech uttered by a person with 
voice-speecli disorders, using a technique used for correcting the voice-speech 
disorders, othW than the speech production substitutes, as the speech uttered with 
voce-speech disorders, to generate speech signals. 

28. A speech transformation method comprising: 

detecting the speech to generate speech signals; 

performing speech recognition processing using speech signals from 
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acousto-electric transducing means; 

working on or transforming the result of recognition depending on the bodily 
state, the using state or the using objectives of a user; 

generating a control signal for outputting the result of recognition and/or the 
result of recognition worked on or transformed; and 

outputting the result of recognition worked on or transformed based on said 
control signal for presentation to the user. 

29. The speech transformation method according to claim 28 wherein the speech 
uttered by a person with voice-speech disorders is detected to generate speech signals; 

the speech is recognized based on speech signals; and 

the speech information representing the speech outputted using speech data 
generated on pre-sampling based on the result of recognition is generated. 

30. The speech transformation method according to claim 28 wherein the result of 
recognition and or the result of recognition worked on or transformed are 
demonstrated as image on display means. 
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(57) Abstract 

A speech converting device comprising a microphone (21) for generating a speech signal by detecting a speech, a signal processing 
unit (22) for performing speech recognition by using the speech signal, a speech information creating unit for processing/converting the 
recognition results depending on the physical condition of the user, the used state, and the use object, a display unit (26) for generating a 
control signal for outputting the recognition results and/or the processed/converted recognition results and presenting it to the user and a 
speaker unit (25), whereby the speech uttered by a person with speech impediment is processed/converted, the processed/converted speech is 
presented to the user, a speech from outside is processed/converted, and the processed/converted speech is presented to the user. 
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struct Generation Function. Proc of 17th Ann Int ACM SIGIR Co 
nference on Res and Development in Information Retrieval, 152 
-9, 1994; Edmundson HP, New Method in Automatic Abstracting. J 

of ACM 16,264-85, 1969) , S d ^ r (D^^lf fg^^rg^ 2 3 fi, M 

FreeiligDP^fflV^Tfi^:3r-'7- K£7)^ftm^fTVN, Incremental Pa 

th Method^ m^^xmmtnm^n ^ ^ t:d^^-^mx^^ (mm Nakaz 

awa M et al. Text summary generation system from spontaneous 




• 
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speech, The 1998 meeting of ASJ 1-6-1,1-2, 1998) o 
(D=^m. T^-ir > h^{c:joV^T. yN*Lfc«9> 

f&^e^gp 2 3 M^«J^>lT(^:S:^^{^iS^$tLTv^§#&^^^E L 

fc^S^^p^tf fg^^oV^T^f 5 Warren RM, Perceptual Res 

toration of Missing Speech Sounds. Science vol 167, 392, 1970 ; W 
arren RM and Obusek CJ, Speech perception and phonemic resto 
ration. Perception and psychophysics vol9, 358, 1971) o 

«9*^2oooHz (DmmcD^p ^m^^^ . Wi^t^^i^ i 

5 d B) J: 9 {-^«^'^m$-ttT^^tf fR^m^ UTt>avv 

r (Z)^^tf 2 Sfi, -|?!l;ti^US PATENT 4628528 (d i ^ 

tl ^ tbXV^S T n — y^-^ 5^ 31 — :/ • 11 ^ K (Acoustic Wave 
Guide) S^T^fflV^TtJy;^ ^nS^Kidiftfa Lfc^Jd^i^ LT^^t* 
m^ffi;^ LT J: < . T = -^^^ o,^ • ;^f-i' K£^#f(cS 

■^^^ 1:= — ;«7;?)^bO^^W(-ii UXm LT J; (^ij wave 
radio (BOSE))„rrT^ W^tf ^g^^lfP 2 3 {l. 

^i^rS :7 ^yl^iS?— :5^0;St^^To-C^?^t*^^tU;^i- 
'5^:S^^ToT t ^ < . '|5iJ;t{^SUVAG (Systeme Universel Verbo-t 
onal d' Audition-Guberina) ^ ffl V ^ 5 r ^ {d J: "9 . Bf^Oj^^S^^ 



• 
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^ffoTti^V^L^ y>^m±^XG works v. 3. 0 (-^-^y^) (D^T^^ 
2 6 (C^f^^^Tj^ LT t> J: V^, ^fc. r (7)#|^tf ^4^^!^^^ 2 3 {i, 
J: 5 {-^^tf ^^^^^ LT^^§I5 2 6 (z:^7J^UT^av^L. 



m*:-e-rJ;j r^iS;^si|o t t ^i- J: j ^ ;^ t- — 2 5 m ;^ 

^ U -r* 3;- ^ 
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T^^^-itSo i^<D—mt\^xn. ^^tf 2 3 mx.i-f 
m^>2 2X(D^^%Em'i^^t ]^x rr>^^-v im^i^Mi^x) J 

JEf'ftilco^iJ t LTI*. ^^X2mm {^:^\^- ^ — ^ v-^ 
tiz. h\^)t^\L. ^^'It^^^gp 2 3 tt. m. 5 {ciTj^-r J: 5 ff^lt^ 

TMj (Dmm^m7jk-r^. mjc:. w^'it^i£^a5 2 3 B^i+isig 
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i- 6 ^ -e # § , 

2 1 tf^ffi L^r^^(D^». ^(^J^^i5:4#t4{^^c; CTM/^6;!jnx^ 

^tf ^^fifelfP 2 3 TMi. -^JX.!**-^^ r2 2 1 7< ^ :^ fh ^ 
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fi§T^^ — ^> 3 ^^S^fi)t^tL5 (mm Ando H et al ^?^-^lg^t^ 
i^^-f'M.(Dr:i^(D'^W^^mmm.^mM<7:>^lii Analysis of speech 
prominence characteristics for translating speech dialog to 
sign language. The 1999 meeting of the ASJ society of Japan 
377-8, 1999) = 

^(Dx^iz, ^pmm±f^^2 3 -^^ u 7i->> 2 1 x^m 
^(D^^(D^iz-D\.^x(Dmwi'^^^^j^i^x^pmm^±!^i.. 

— 2 5 S:t>VX}*^^a5 2 6 (C^^-f ^ r i: -C^l^ffi^d^P Pj-^i: 
T^av^, -ft-^^S^B 2 2:SLt>"#^'l*fB^fig§P 2 3 I*. 
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IB'IS^. 2 4 (^^^1h>:^y ^/i^^^LT^^^rt Lfc^^T^'-^ ^^^tf $B 
^^gp 2 3 Ci J; <9 m^^^^ii:^ :i ^ {' J; ^ #^'tf U T {±J;^ 

^ -e ^ s o 

Lfc^^. lElt^Il 2 4 {dii, 0(D=^^^^—^ (D^ti h-r^^cD^^ 

,2. JL A- At i-l^- Vrl7 <-»/->)-!- 3te ^=i=- At SB /^^ rtr.l .i. BB /-rz 

w <_ c N B iH Ti>t ;-tXw rp o I ^ -tr e V — v |^ -a: pry ^ , |^ -i^p 
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(^ij : STRAIGHT) ^Jg-T ^ ^ j; 19 . f- :7= y L Tlfilt 
Mi-^/'^ 'ft-^^S$P2 2f*. sB^tEWi (speaker recognition) 

^ A;^ $ S ^ ^ {c: o V ^ T =fT o r # IS # {;i >5tf j^; L fclSfli*^* ^ 
^^LTt3^V^. ^LT. ' (Dft-^^a§P 2 2 T-ii. ^IS^f-Mt- 
5tf t h {c>?> t- — ;^7^. 2 5 -^m^U 2 6 CitU;^-^^ 

"C^i&l''' Soong FK and Rosenberg AE, On the use of inst 

antaneous and transition spectral information in speaker rec 
ognition. Proc of ICASSP' 86,877-80,1986) = ^(D^^ h/I^fi^-fb 

(7) ;^ ^ .ffl V ^ T # m ^ HS ^ (7) fIJ ^ ^ 9 = 

fcoTtj^V^ Zheng YC and Yuan BZ, Text-dependent speak 
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er identification using circular hidden Markov models. Proc o 

f icAssp' 88,580-2,1988) . ^(D:)^m-vit. mmm^cDinim t 

t LTleft~to~right^7=Vl-{^^ LT^/P=i-'7=V ^ f;^HMM^m 

Mll^ 1 -ef±. ATR-MATRIX (ATR^^SIRii 
itW^PjfM : Takezawa T et al, ATR-MATRIX: A spontaneou 

s speech translation system between English and Japanese. AT 
R J2, 29-33, Junel999) "effl V^ b tbX § W^ISI^^S (ATRSPREC) . 

^^^}^mm (CHATR) . mmmm^m (tdmt) ^n^:itx. -^^ 

#^S^>ISs^S (ATRSPREC) -eii. i^m^mm.'^^m^^n\^^ (ma 
ny-word continuous speech recognition in real time) ^ e^f^SE 

my-7i^^m\^^x=^^mm\^'^^mfj:^m'^'f^i^hmm=^7'^7V(Dm^. 
'Si.xi\t^^mf)^hW.m'^x(DT.'^^^m-r^. ^^D^^^m^^'^x 

IS. n^fz.^'m-^y~-7l^mt LT^^L (complete group of tool 
s) , y—7\yW\±<Dm^^f^^^:Ltf)^^^X (easy integration 
of tools) S-I^^H T K <5r (Z)5i^tt$tf^ (compatible with HTK) ^fy 

^ -il -h^ ^ /T\ Ofc ^ ^T} Hit -ir, :X 1^ -X- rtdt nbr- db^ ^ is^tL ^=fcs- 5^ Siih _ 

^ c X ^ ^-^ H RiQ-^ PH);. '<L I J y <^ ^ /vC pd -iH ^> e pic^ '<i: 1 J 

(ATRSPREC) nm^^^j:'^^%E-^miiii!m(7:>-d^ti t \^x 
&.T(D (a) ~ (d) iz.^.-r X 0 ^^j^y-zi^m^^m-r 
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^mm^m (atrspreO i^-unix^^ (osfi,hp-ux) x-mi^-t^o 

_(a) m^^m: xr^^^im^Lti^p<D^mm-^^i>^'i^^^ h/\^ 
(b) '^m^'r/i-mm •. mw^^^ h/i-^f^rt^(DM#.*^^°^;'f — 

HMnetCDft^ (MLtt^fe, MAPfi^j*, VES, MAP-VFS) ) o 

mmmm'^m (tdmt) x\±^ -X(Dmm^'¥m-r^^m. ^mmm 

^^-a-Sfe^S (CHATR) X\-X. hht^\:.^^^-^-<~:^it^M^^ 
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^p^m^m (CHATR) tti^T(cTt#fife$tb5. m^^^m^—:^^ 
^^sim^s. mfgsf^^a. ^^^fife^a, x 

^^If fg^tfifegp 2 3 It. (ATRSPREC) {:::J6^^ 

-Z^"" -7^^ '/^> tW- ^01 FR /Til mjx-^^ >- l^)-*- Uln ^ > J.* >.i- J. 

« / ^ N N BP rn i7'j '<L m V • ^ c tiw V ^ pi^i Itr ^ \ /C U /C 
fg^)5fe$P 2 3 It. -v-f n 2 1 -r#lg4'cD-^^SK#^tL/=?/^^ 



m 
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^fc. ^^mn^^^2 3 ^P-B-j^ (chatr) ^c:io^^T. f-i^ 

^~^i\:vxm-m^nt'^A(D'^^miiL(D^^v'-^ ^^h. 

^§[5 2 6 (dtti;^ LT t ^^/^, ^SflE^llfi. ^i^y'/i^y< 

^ttfR^^fife LT ^3 (#Be Ishikawa K, Sumida E, A comput 

er recovering its own misheard-Guessing the original sentenc 
e form a recognition result based on familiar expressions- 
ATR J 37, 10-11, 1999) = ^ (D t ^ . ^F'lf ^^fife^ 2 3 li. ^.M^ 
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±iz!! LfcM^,^ 1 (7)S^PJ{c:joV^T^i. ti >- 2 l T 

§ J: 5{-it-^^S§P 2 2 {cij; ^9^^^LTti^v^o ^ ^<^)^f^A 
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« 2_9 2 {CTj^-r J: 9 i^MPi. 2 6 {c^^-rs tjc^-e^o 

Ttj^V^c :^ ^^«2 9 iif?ij;ttf|ll 1 (Z) ^ ^ lf|5 7 i 

r CD J: 9 J^^Mfli^ 1 i*. ;t ^ =7 2 9;6^p3CPU^^^f> 

sm^^smss^M* L-c^^gp 2 6 izmmvtcmm^ myj^-i- ^ o 

^ 2 9 xmrni^tzMi^^At) LTilfttilUg 2 7 ^I>^{t^^ 

g|3 2 2 i^f^lf.lS5^S^fTo-C^^tf fg^^gP 2 3 
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Wt^^ ^W^^m\^(Di^Wi^'^-V. Karhunen-Loeve (KL) 

(DK h mmi:i X ^7jk7L!Emiz.m-^ < k i iitmj^j:^ ^(7ym^^<i^ - 

^ t^^-^fi-^xL-s ^ ^ "9 . f^ffl#{^i: oT^J-Ig^idpr^i- 
^S(iJ^XT(7>:i:^{^te«t^nrv^§ (Kosugi s, ^ — ^/p^. 5/ 

ffi^^fcM®^(^)ltBli ^ ^mttffi tf^^^CVW^, 73-2,1991-07; T 
urk MA and Pentland AP, Face recognition using eigenface. Proc 
CVPR, 586-91, 1991-06; Akamatsu S et al, Robust face intificat 
ion by pattern matching Based on KL expansion of the Fourier 

GJ et al. Learning to identify and track faces in image segue 
nces,Proc of FG '98,260-5,1998) , 
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mi\^M-t^%^^W.t^^-tc :i CO^f^lf^lt^S^Ubiquitous Talker 
(Sony CSLM) Tffl V ^ e> tl.T V ^ S tf ^ ^ffi "T 5 ^ ^ ;e>5 "C # 5 (# 

% Nagao K and Rekimoto J, Ubiquitous Talker : Spoken language 
interaction with real world objects. Proc 14th IJCAI-95, 128 

4-90, 1995) c 

#iJx.flMPEG (Moving Picture Experts Group) ^ ffi V ^ 

m^mm-r^ :itx\ fg^^lS*cDPW^»f*LT^^^P 2 e 

tx. mm^mi^xmw f^-^m t nm^^ i^^k^Lx ^ r 

r t xm^^mmL±i:^(D^mn.^-o:d^}^^pmm<7:>iE 

5ft$/j^f^±L. Mf-ijcS (lip reading) ^ n 0 ^ t t fj: ^ W- 
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^ij^^p^tg^ L-c ^avN„ ^(Dh^:^^ y^mM\±. '^m^iz.m'^-^ 

^ ^ 2 1 \^fz.^^^y^^ — tiU 2 5 (31 J; 19 ]^f}-^^ii 

"jmm 2 9 J: 19 tift LfciH^^^^^^fn 2 6{c:J:'9fcB;^-r-5J;9 

JCfc RH »-+ 1^1 -ir I OR -4- LlJ -r -MK +lfe ) -.- PR f -r 1 . ij. J, , , ~ i . 

'-t~ :7ci -yj 1=*, > >iA I «>- P7L -Tl ? 'oJ M-J yj V 'oJ T^lffi- *^ RIX O 4 1/ -oJ ^ <:r tct ' ^ C 

i~/^:b-t,. r (Z)M^.^i {c:*5V^T. "^^mn^ iHtiir :5mm t 
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(#BS Sugiuchi T, -fr^M^^^CDiifS^ i JOHNS Volll No9, 130 

4, 1995) ^{fx.. ^^tffg^fifegP 2 3 id j; 19 ^i^-TS r t tci J: ^ 

^^^^;\^=r.^ K (Tactile Aid) ^60^1: J; § Mftg^^f ^fljffi L 

^^iM^LT;^t=-;^7$p25^^^tti;^-t-'5^^^m;^LTt>Sv>, 

7J^ L T ^> B:V^, 
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-f' ^^^^m^^ l^-lf^ly h (Auditory Brainstem Implant : AB 

Mj^^fc. ro^isfiifi, ^m^cD^i^^^m. mmvtm:sL-(jm. 



iSttjJ^mtff (bone conduction ultrasound : Hoso 



i H et al Activation of the auditory cortex by ultrasound. La 
ncet Febl4 351 (9101)496-7.1998) ^ffi V^T;S^iiSJl ifeiS:^^(D{t 



-ir^ -7. 4^ JH^ -r- m 1 . N -7^ f±i >" 



>V ULI re CfL rj-, >^ Xrfi -rr. 

(f^ vc*. tr U-i yj -^-455. 'd: <ja; v. 



-4-, 1 ^ 



^ # i± ^ ^ 1: ^ ii: 5 5fe (7) §^ (D rng ^ ffl V ^ T ^ ^ ^ au 
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v^Tt>^< . 1S^lHj{^WoTffi;^)ISm^tti;^-r5:^^-lH](::p;SoTtB 

^^$QiS?rtTo T*7F§P 2 6 IZ^^-^^^W, 1 (D^m^n ^ CPU 

:^^- — tiU2 5 {^m;^*§m^ffi;^-t-S/cfccD^ 2 (^:!!0;S^fT 9 C P 
U i: , ^ 2 9 -C^lt LfciUft^S^-rS 3 CO^S 

^m^n V c pu ^mmi^mi'^^'ttxm KD^^m. m2(Diin!m. r 

(D^m. m 1 RXfm 3 2 RXfm 3 <7)^S^fT 9 C P U 

% IS] { ift $ -It T m ;0 ^ it T t> ^ V > , 

m^^^^ti^xmtjLx-r^ x o i^^^mn^i^u 2 3 xm^l 

MJ::. r 1 It. ^mocPU^^L. ±3z!! L li^O C 
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c p v-xrff 9 i t {d. comim^m(DC p u-rfro-c ^ ^v^ = 
m:tl-£. r cDfflli^ 1 ^::ioV^-c. -O c P u a;^ ^ ix;^c^ 

(text to speech synthesis) ^^lott'hi^. Xfit>^oC0CP 

(7) C P U ^)^^ Kt} tlfzm C^^iClx* LTSTRAIGHT^JS^^To ^ L 
T t^ — 2 5 ^:ltii;'Ji-5^({lS^^TV^, ftii© c P u^sAyb ^fht^ 

fro TJiizEcofll-ir (7:)t±i;'jf^«{3.m;^'r5 ^S^fT 9 c pu^^-T^ 
t t-hiz.^ ^inx'^m^S^^fe^^^v^-c-v-^ ^ n ^Ni/ 2 1 (ciA;^ 

n 9 fz^CDC P\J t . \'^(D-mJL^^^m-^f7 9 C P U i ^S>J#l(-{i 
15?: y 'cj xiJ^'d: 1 J V c c t> *«- N w:7<<:'^>' s >tv yv_i-»KicS^ war /tj vcr\./tt 56 



• 




wo 00/49834 



PCT/JPOO/00872 



47 



2_3 -CtT 9 ^S^. m:^lty — V ^^'^-iJ^'^H (STRAIG 

LTHMD, H^PlS-a^M^^i^B (Head-cou 
pled display) ^ AXIS (visual prosthesis/artificial eye) fj^ 

feifbnso ^:xTi^m^^'r ( (a) ~ (m) ) . 

(a) mm^HMD (:^^m^\z.n^m\m^m^\^yLmM.^'^m 

( b ) ^Ig^HMD 

(c) — — MHMD . ±lZAR^m^-t ^ i^(D t i^XEye- 
through HMD (Puppet Eyes : ATR) 

(d) u'Mimm'^UM.^mmmn^v'-^ 

(e) mmm(DMmmmm i^mmMmmH. ^<~^^ ^ ;v 

^— (Visual filter) ^fflV^-5) 

(g) (Virtual Retinal Display, Retinal project 

ion display. mm^mmcD^Wim 

(h) AXBS (visual prosthesis/artificial eye) "l^^l-^ 
Tjijftx — ^ ^f^efe b> f$:F^ilJs63A^OME N S (Micro-Electric 
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al Mechanical system : ft-?- [a] ^{i X. -v n -v- > ) ^MWi 

• ^^-cmi^T"—^ tMEN smw}m(Dm:^^m.i^-r^-. mens 

o< <9 b/^Oft-^^fiJ^mi^^ii AX 
Bgt'f^ME N S ^SfejAtf^Eff (c: J; t9 hl~h4(;i^l-t btl-So [hl]Hul 
I^^IJ^SJAXBS (cortical implant : #BS, Dobelle Wm H, Artifi 
cial vision for the blind by connecting a television camere 
to the visual cortex. ASAIO J 2000:46,3-9 ) Lh2'\ mMMWcWl A 
XBg (Sub or Epi -retinal implant : #BS Rizzo JF et al. Deve 
lopment of an Epiretinal Electronic Visual ProsthesisHarvard 
•Med MIT Res Program. inRetinal Degenerative Diseases and 
Experimental Theory Kluwer Academic Plenum Publishers, 463-7 
01999) , Ch2]^^*^$IJ;^§^AXBS (optic nerve implant : #as 
Microsystems based visual prothesis MIVIP (Cathol ique Univ S 
ci Appliquees Microelectonics Lab) ^ [h4]^^^ U 5/ KMAXIH 
m (hybrid retinal implant : iHJSSitft+fflfliflJ'^M A XBg Nagoya 
Univ) ^^^fc-So 

(i) W^AtimmU^UMU (HAQ-200 (S^S^f^Bfr^) 

(j) sisism^K (:?. m. mm. mm 

head-mounted projector : linami m el al. , Head-moun 
ted projector (II) -implementation Proc 4th Ann Conf Of Vir 
tual Reality Society of Japan 59-62,1999) ^ ]) ^ ^(D^lW-'f 
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(1) i^WiMT' ^ (spatial immnersive display) 

omnj^max^ CAVE Cruz-Neira C et al. Surrounded-screen 

projection-based virtual reality: The design and implementat 
ion of the CAVE, Proc of SIGGRAPH' 93, 135-42, 1993) , CAVEMaI 
i^^^M^^M (CABIN:#BB Hirose M et al. lEICE trans Vol J8 
IDII No5, 888-96, 1998) , ^l^MmitiUWy^' ^ ^ ^ ^ 
^l^-^ im : CAVE)SI/HMD #86 Endo T et al. Ultra w 
ide field of view compact display. Proc 4th Ann Conf of Virt 
ual Reality Society of Japan, 55-58, 1999) ^ T— V—l^) 

(m) ^(Di\iL h :/BgM (Upton eyeglass) (7)7=" ^' I-' -< V 

— ^ LT ^ J: V^ ( 3 DW#i/>?.7" AISHead-Related Tr 
ansfer Function ^ffl V ^ fc^r^l #i!^.^fi£ v- :x "T" ^ ffl V ^ ^ : ^^ij Con 
volvotron & Acoustetron 11 (Crystal River Engineering) ; ^4 
-j-^ -/^MK^^v<j3.-.^ h^mix^ Yu V-^ 4 -^^ TJ -7 ^ 
ffl Lfeffili^TE-H50 (Sony) ) , ^ ifiV^^m ^ O < o »9 . b 

s^Tis 3 Bmi^W^i^^n ^ CAVE izMftir^) ^ffl v^fc «9 "T -5 t> 
3!{^*fci, _b5z!i(7)HMD 2 ii, lSTI§|5{:i 3 ?j^v7c{iait ttiir >'i^ — 
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^l^m^Ml^ (Augmented reality: AR) ^ fij ffl L fcl^il^^t5 1 T? . 

n Tjsi/ 2 1 -v^^in L^^mm±f^^> 2 3 -c±f^Lti^pmmt ^ 

y( iyP^TJ^lZXK) — /\yV T V T ^ (Virtual reality: V 

ir >'-y- 3 1 i LTlSg|5(:i/jNMi>>'l' n -fe ^'-t^ (^'— ^ ^ o, gU-301 

1) m.m^<j:>i^\^DrM^^i^-^ — ^ ^ ^ ^ q^i-zq\2) ^mm 

i- -1) o i^iiCDir > It 3 1 fy^ o ^ b F tf 2 3 X'-^^ ^ 

=fTo fc^, i^m^(D:^ . {^>?ti^i-'5 v-'^-iS^ 3 2a, 

3 2b T^S^ LXa^gP 2 6{^ll^ti{fe;iS^T< i:V^9V'><.7"A^fflV^ 
-5 r i -e Ritt i: ^ o (#B^ Ban Y et al, Manual-less operation 



• 



m 
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with wearable augmented reality system. Proc 3th Ann Conf of 
Virtual Reality society of Japan, 313-4, 1998) „ 
i^i.TO:^-?i-e^iAR^m^tfe-t?feS. :^ ;^ V7()^h(Dmi& (video St 
ream from camera) J: «9 — :^ — ^ — 5^ (search for marker) , 
■^ — tl(D 3 DfiE • :fj\p]^^-r>n (find marker 3D position and 
orientation) ^ — (identify marikers) ^ i/ 3 >- 

t:^y'iy::^^ h CO/Ki/i^ 3 (position and orient object 

s) , t"^';^-T*cD 3 D:^:/v?3^ h^^fiJcL (render 3D objects i 
n video frame) , v"^ HMD iZtH^b ir ^ (video stream 

to the HMD) : Integrating real and virtual worlds in share 
d space. ATR MIC Labs and HIT Lab, Univ of Washington) ) . 
^fd, ^co^Jiit^ 1 '^-^ 3 1 \z.m:^X^UmWit^ 

(M : Ubiquitous Talker (Sony CSL) ) t V R ^M^-T ^ 

1 t ^t^p6tf^-fflv^s ^ ti^x'o . A R ^mt-r^ r t ^ wt^trfe 

irVi^S 1 (tf fgA;^ v'^'^-T^i^) ^ LTl^TcDy=V-^^ 

3 JSJ^TcflLgir (ExpertVision HiRES & Face Tracker (Mo 

tionAnalysis) ) , 3 ^Tcfifiir (InsideTrack (Polhe 
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mus) , 3SPACE system (POLHEMUS) , Bird (Ascension Tech) ) , 
5t 3 D -r"^ ^ i?' ^ if (MicroScribe 3D Extra (Immersion) ) , 
^^^3 Dt'V -< (Model350 (Polhemus) ) , =^^^3D^' 
^i/^-Y "tf (Sonic Digitizer (Science Accessories) ) ^ 
3 r> ^ -r— (3D Laser Scanner (T^7^5/i7>^) ) , ly 
if (i$:iH(Dm^X^m^) y< — -7 4 l^iS— (N T T t 3.— -v-i/^ 

:7^— >!.^^^) ^ ^mW.'fy^^:^ (DetaGlove (VPL Res) , 
Super Glove (0^^l^^ Y- xj. =. ^ ::^) Cyber Glove (Virtual Tec 
h) ) , -7 :^ — :^-7 ^ — Y /<. ^ (Haptic Master ( 0 1^:11 \^ ^ y 
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